Large Language Models (LLMs) are increasingly used as proxy students in the development of Intelligent Tutoring Systems (ITSs) and in piloting test questions. However, to what extent these proxy students accurately emulate the behavior and characteristics of real students remains an open question. To investigate this, we collected a dataset of 489 items from the National Assessment of Educational Progress (NAEP), covering mathematics and reading comprehension in grades 4, 8, and 12. We then apply an Item Response Theory (IRT) model to position 11 diverse and state-of-the-art LLMs on the same ability scale as real student populations. Our findings reveal that, without guidance, strong general-purpose models consistently outperform the average student at every grade, while weaker or domain-mismatched models may align incidentally. Using grade-enforcement prompts changes models' performance, but whether they align with the average grade-level student remains highly model- and prompt-specific: no evaluated model-prompt pair fits the bill across subjects and grades, underscoring the need for new training and evaluation strategies. We conclude by providing guidelines for the selection of viable proxies based on our findings.

Indeed, there is already existing evidence to suggest that this might be the case (Bommarito and Katz, 2022; Choi et al., 2023; Kung et al., 2023; Frieder et al., 2023). However, apart from one study by legal scholars (Choi et al., 2023), existing evaluations on university exams probe the model only on a subset of the task for which it might be particularly suited (for example, excluding all questions that contain images). In addition, evaluation of the model's responses is often not blind, which can be problematic because ChatGPT is known to produce strange answers that are subject to interpretation. As such, despite much discussion about the topic, there is to this point little systematic evidence regarding the capabilities of ChatGPT on university exams (Mitchell, 2023). We present the results of a simple but rigorous experiment that evaluates the capabilities of ChatGPT on an undergraduate computer science exam about algorithms and data structures. We conducted this experiment alongside the regular university exam, which allowed us to evaluate the model's responses in a blind setup jointly with those of the students. We posed the different exam questions in a simple standardized format that allowed ChatGPT to give clear answers to all exam questions.

One common problem with machine learning algorithms is that we don't know how much training data we need. A common way around this is the often used strategy: keep training until the training error stops decreasing. However, there are still issues with this. How do we know we're not stuck in a local minimum? What if the training error has strange behavior, sometimes staying flat over training iterations but sometimes decreasing sharply?